On Embedding the Angles Between Signals

The phase of randomized complex-valued projections of real signals preserves information about the angle, i.e., the correlation, between signals. This information can be exploited to design angle preserving embeddings, which represent such correlations. These embeddings generalize known results on binary embeddings and 1-bit compressive sensing and reduce the embedding uncertainty. 2013 Signal Processing with Adaptive Sparse Structural Representation (SPARS) This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c © Mitsubishi Electric Research Laboratories, Inc., 2013 201 Broadway, Cambridge, Massachusetts 02139 On Embedding The Angles Between Signals Petros T Boufounos Mitsubishi Electric Research Laboratories, Cambirdge, MA 02139, USA, [email protected] Abstract—The phase of randomized complex-valued projections of real signals preserves information about the angle, i.e., the correlation, between signals. This information can be exploited to design anglepreserving embeddings, which represent such correlations. These embeddings generalize known results on binary embeddings and 1-bit compressive sensing and reduce the embedding uncertainty. Randomized embeddings play an increasingly important role in signal processing applications. Such embeddings transform a signal space to another of typically lower dimension or convenient computational properties. The embedding approximately preserves some aspect of the signal geometry, such that operations on the embedded signals directly map to operations on the original signals. The most celebrated embeddings are Johnson-Lindestrauss (J-L) embeddings, preserving l2 distances [1]. They are functions f : S → R K mapping a finite set of L signals S ⊂ R to a K-dimensional space such that, given two signals x and y in S, their images satisfy: (1− ǫ)‖x− y‖2 ≤ ‖f(x)− f(y)‖ 2 2 ≤ (1 + ǫ)‖x− y‖ 2 2. In many applications, however, l2 distance is not the appropriate distance metric. In this work we explore embeddings of signals that preserve angles, i.e., correlations. The remainder of this development uses the normalized angle defined between two signals x and x’ as d∠ = 1 π arc cos 〈x,x′〉 ‖x‖‖x‖ (1) Often, angles between signals are more informative than distances. Furthermore, if signals are normalized, their angle captures their distance. Embeddings preserving angles instead of distances can be more efficient in such cases. Angle embeddings were first introduced in the context of 1-bit compressive sensing [2]. The binary ǫ-stable embedding encodes signals using a random projection followed by a 1-bit scalar quantizer that only preserves the sign of each projection coefficient: q = sign(Ax). (2) The normalized angle between two signals x and x′ embedded in q and q′, respectively, is preserved in the normalized hamming distance between their embeddings, as follows: |dH(q,q ′)− d∠(x,x ′)| ≤ ǫ, (3) where dH(x,x ′) = ( ∑ i xi ⊕ x ′ i)/K denotes the normalized hamming distance between the signal embeddings. Instead, in this work we consider continuous embeddings, obtained by first projecting the signal to a complex-valued space and only preserving the phase of the projection coefficients: y = ∠(ACx), (4) where AC ∈ C K×N is a randomly drawn matrix with i.i.d. elements drawn from the standard complex normal distribution. To demonstrate that angles are preserved we should show that given a pair of signals x, x′, the expected value of the phase difference of their complex projection coefficients is proportional to their angle E { ∣The phase of randomized complex-valued projections of real signals preserves information about the angle, i.e., the correlation, between signals. This information can be exploited to design anglepreserving embeddings, which represent such correlations. These embeddings generalize known results on binary embeddings and 1-bit compressive sensing and reduce the embedding uncertainty. Randomized embeddings play an increasingly important role in signal processing applications. Such embeddings transform a signal space to another of typically lower dimension or convenient computational properties. The embedding approximately preserves some aspect of the signal geometry, such that operations on the embedded signals directly map to operations on the original signals. The most celebrated embeddings are Johnson-Lindestrauss (J-L) embeddings, preserving l2 distances [1]. They are functions f : S → R K mapping a finite set of L signals S ⊂ R to a K-dimensional space such that, given two signals x and y in S, their images satisfy: (1− ǫ)‖x− y‖2 ≤ ‖f(x)− f(y)‖ 2 2 ≤ (1 + ǫ)‖x− y‖ 2 2. In many applications, however, l2 distance is not the appropriate distance metric. In this work we explore embeddings of signals that preserve angles, i.e., correlations. The remainder of this development uses the normalized angle defined between two signals x and x’ as d∠ = 1 π arc cos 〈x,x′〉 ‖x‖‖x‖ (1) Often, angles between signals are more informative than distances. Furthermore, if signals are normalized, their angle captures their distance. Embeddings preserving angles instead of distances can be more efficient in such cases. Angle embeddings were first introduced in the context of 1-bit compressive sensing [2]. The binary ǫ-stable embedding encodes signals using a random projection followed by a 1-bit scalar quantizer that only preserves the sign of each projection coefficient: q = sign(Ax). (2) The normalized angle between two signals x and x′ embedded in q and q′, respectively, is preserved in the normalized hamming distance between their embeddings, as follows: |dH(q,q ′)− d∠(x,x ′)| ≤ ǫ, (3) where dH(x,x ′) = ( ∑ i xi ⊕ x ′ i)/K denotes the normalized hamming distance between the signal embeddings. Instead, in this work we consider continuous embeddings, obtained by first projecting the signal to a complex-valued space and only preserving the phase of the projection coefficients: y = ∠(ACx), (4) where AC ∈ C K×N is a randomly drawn matrix with i.i.d. elements drawn from the standard complex normal distribution. To demonstrate that angles are preserved we should show that given a pair of signals x, x′, the expected value of the phase difference of their complex projection coefficients is proportional to their angle E { ∣